Eyewear-mountable eye tracking device

ABSTRACT

An eye movement tracking device that can be mounted to standard eyeglasses as disclosed. The device comprises an illumination source, a time-of-flight (TOF) camera and a processor. The source transmits energy within a frequency band from a location proximate to an eye of a person, such that at least a first portion of the transmitted energy is reflected off a lens of eyewear worn by the person to subsequently reflect off the eye, and such that at least a second portion of the transmitted energy is transmitted through the lens to reflect off objects in the person&#39;s environment. The TOF camera detects reflections of at least the first portion of the transmitted energy, and distinguishes them from other energy detected by the TOF camera in said frequency band, based on TOF principles. The processor uses the detected reflections of the first portion of the transmitted energy to determine eye position.

This is a continuation of U.S. patent application Ser. No. 14/542,455,filed on Nov. 14, 2014, which is incorporated herein by reference in itsentirety.

FIELD OF THE INVENTION

At least one embodiment of the present invention pertains to eyetracking technology, and more particularly, to an eye tracking deviceusable with standard eyeglasses.

BACKGROUND

Eye tracking (also called gaze tracking) technology is evolving tobecome an important part of next generation human-computer interfaces.Eye tracking technology has many potential applications inentertainment, research, and as an interaction tool for people who arephysically impaired.

Most known video-based eye tracking systems use infrared (IR) cameraswith IR light sources to detect the pupil/iris as well as glints fromthe illumination. Additionally, these systems generally use eitherdirect imaging or indirect imaging. Direct imaging systems image the eyeregion directly by placing one or more IR sensors directly aimed at theeyes. These systems have problems, however, with occlusion of the sensor(e.g., from eyelashes) as well as the user's vision (e.g., from thesensor). These systems have particular trouble tracking the eyes ofpeople who wear eyeglasses, at least partly because the eyeglasses causeocclusions or disturb the computer vision algorithms by creating falsespecular reflections.

Indirect imaging systems partly address the occlusion problem by viewingthe eye via a so-called “hot mirror” lens, which acts as a mirror tonear-IR limitation but as pass-through glass to visible light. In thatway, the sensor can view the eye from a frontal position while avoidingocclusion of the user's field of view. However, the indirect imagingapproach requires special glasses (i.e., equipped with hot mirrorlenses) and, therefore, is not suitable for people who depend onstandard (prescription) eyeglasses.

SUMMARY

The technology introduced here includes an eye tracking device that canbe used to perform eye tracking for a user wearing standard eyeglasses.In at least some embodiments the device comprises an illuminationsource, a time-of-flight (TOF) camera and a processor. The illuminationsource transmits energy within a frequency band from a locationproximate to an eye of a person, such that at least a first portion ofthe transmitted energy is reflected off a lens of eyewear worn by theperson, to subsequently reflect off an eye of the person, and such thatat least a second portion of the transmitted energy is transmittedthrough the lens to reflect off objects in the person's environment. TheTOF camera detects reflections of at least the first portion of thetransmitted energy, and distinguishes the detected reflections of thefirst portion of the transmitted energy from other energy detected bythe TOF camera in said frequency band, based on times of flight of thereflections. The processor is configured to use the detected reflectionsof the first portion of the transmitted energy to determine a gazedirection of the eye of the person. Other aspects of the technique willbe apparent from the accompanying figures and detailed description.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the present invention are illustrated by wayof example and not limitation in the figures of the accompanyingdrawings, in which like references indicate similar elements.

FIGS. 1A and 1B show perspective views of an eye tracking deviceaccording to an embodiment of the technology introduced here.

FIG. 2 is a block diagram showing functional elements of an embodimentof the eye tracking device.

FIG. 3 shows an example of the eye tracking device mounted on a pair ofstandard eyeglasses.

FIG. 4 schematically illustrates the propagation of IR illumination in atypical usage scenario.

FIG. 5 illustrates an example of an IR illumination scheme for anembodiment with two illumination sources.

FIG. 6 illustrates an embodiment of the eye tracking device with twoillumination sources.

FIG. 7 is a flow diagram showing an example of the operational processof the eye tracking device.

DETAILED DESCRIPTION

In this description, references to “an embodiment”, “one embodiment” orthe like, mean that the particular feature, function, structure orcharacteristic being described is included in at least one embodiment ofthe technique introduced here. Occurrences of such phrases in thisspecification do not necessarily all refer to the same embodiment. Onthe other hand, the embodiments referred to also are not necessarilymutually exclusive.

The technology introduced here includes an indirect imaging,eyewear-mounted eye tracking device that can be used to perform eyetracking for a user wearing standard eyeglasses. Note, however, that thedevice can also be used with other types of eyewear, such asspecial-purpose eyewear or headsets (e.g. for gaming orvirtual/augmented reality applications). The device can be embodied asan add-on product that detachably mounts to standard (e.g.,prescription) eyeglasses and that does not occlude the user's field ofview. The device applies principles of TOF to its own IR illumination,to distinguish between desired IR energy (e.g., IR transmitted from itsown source(s) and reflected off the eye of the user, which it uses foreye tracking) and undesired IR energy (e.g., ambient IR energy and IRthat has been transmitted from its source(s) and reflected off objectsin the user's environment).

In at least some embodiments the eye tracking device includes one ormore IR illumination sources to emit IR energy; a gated “fast shutter”IR TOF camera (hereinafter simply “IR camera”) synchronized with the IRsource(s) to detect the emitted IR energy; a human-visible spectrum(e.g., red-green-blue (RGB)) video camera (“scene camera”) to generatevideo of the general direction in which the user is looking during eyetracking and thereby enable marking of the user's point-of-regard (POR)on that video; a memory to store image data collected by theaforementioned cameras, a processing unit to compute the eye position ofthe user based on outputs of the IR camera (or at least a communicationunit to enable communication with such a processing unit locatedexternal to the device); a housing to at least partially contain theaforementioned elements; and a fastener compatible with standardglasses, by which to mount the housing detachably to the frame of theeyeglasses.

In some embodiments, the device uses the fast-shutter IR camera, whichemploys an optical focusing system, to collect IR light reflected fromnearby distances, namely, the user's eyes, and to avoid collectingambient IR illumination and reflections of the device's own IRillumination from objects in the user's environment. This can be done bysetting the shutter timing of the IR camera so that IR light transmittedfrom the device will be cut off by the shutter (which may be a purelyelectronic shutter) on its way back to the IR camera's sensor, so thatonly IR reflected or originating from objects from very nearby thesensor (e.g., within a few centimeters) is captured by the IR camera;that is, only energy with a sufficiently short TOF is allowed to becaptured. This enables the IR camera to capture only the image of theeye, without reflections from outside objects. This feature, therefore,avoids the need to use a hot mirror. The IR camera is a “fast shutter”camera in that it can capture multiple images at nearly the same instantin time, by closing and opening its “shutter” (e.g., an electronicshutter) repeatedly within a very short period of time, e.g., every fewnanoseconds.

One type of IR camera that can be used for this purpose is a “gated” TOFdepth camera. A gated TOF depth camera utilizes a series of light pulsesin order to illuminate a scene imaged by the depth camera. For eachlight pulse, the depth camera is “gated” ON for a particular exposureperiod before being “gated” OFF again, thereby imaging the scene duringthe exposure period. A distance to a feature in the scene may bedetermined from an amount of light from the transmitted pulses that isreflected by the feature and registered by a pixel of the camera sensorduring the exposure period. A pulsed IR TOF depth camera of this typecan be used. Alternately, a TOF camera that uses the principle of phasemodulation may also be usable for this purposes of the techniqueintroduced here.

Additionally, the device can acquire two or more such IR images atnearly the same time using different illumination settings, to filterout ambient IR illumination and/or to identify different specularreflections from the eye (glints). This feature enables the device toperform ambient-invariant eye tracking, i.e., allows it to functionproperly in environments with uncontrolled ambient illumination. Thedevice can also compute depth (using TOF principles) for use in gesturerecognition and/or other applications.

FIGS. 1A and 1B illustrate the eyeglasses-mounted eye tracking deviceaccording to at least one embodiment. As shown, the eye tracking device1 can be mounted to the frame of standard eyeglasses 2 (FIG. 1B), suchas to either one of the two temple frame pieces 3, proximate to one ofthe user's eyes. “Proximate” in this context means within a fewcentimeters, such as less than about 5 cm. Note that the physical shapeof the device 1 as shown in FIG. 1 is just one of many possible examplesof the shape the eye tracking device may have. As discussed furtherbelow, the device 1 has a housing 4, which at least partially contains aforward-looking human-visible spectrum video camera 5 (e.g., an RGBvideo camera), at least one near-IR illumination source 6 and a gated“fast shutter” IR camera that includes an IR sensor 7. The devicefurther includes a fastener 8 (e.g., a spring-loaded clip, Velcro, orthe like) to detachably connect the device 1 to the eyeglasses frame 3.The fastener 8 can be a “universal” fastener capable of mounting thedevice to any standard eyeglasses, or it can be designed specificallyfor a given model or manufacturer's eyeglasses.

FIG. 2 is a block diagram further illustrating showing these and otherelements of an embodiment of the eye tracking device according to atleast one embodiment. As illustrated the device 1 includes a powersource 21 (e.g., a battery), a central processing unit (CPU) 22, memory23, a gated “fast shutter” IR camera 20 including the IR sensor 7, theone or more IR illumination sources 6, the human-visible spectrum videocamera (“scene camera”) 5, and a communication unit 24. The device 1further includes lenses (not shown) associated with the IR cameras.

The scene camera 5 can be any conventional video camera. The IR camera20 can be a fast shutter gated TOF IR camera that has a resolution inits depth measurements to within a few centimeters. Note that in someembodiments, the IR camera 20 may include its own processor and/ormemory (not shown), separate from the CPU 22 and memory 23, forperforming image capture and/or image processing operations.

In some embodiments, the CPU 22 controls operation of the othercomponents of the device 1 and determines gaze direction or performs eyetracking computations related to gaze determinations. The CPU 22 can beor include any known or convenient form of processor and/or controller,such as an appropriately programmed general-purpose microprocessor,special-purpose microprocessor or digital signal processor, programmablemicrocontroller, application-specific integrated circuit (ASIC),programmable logic device (PLD), or the like, or a combination of anytwo or more such devices. Further, if the CPU 22 is a programmablemicroprocessor, it can be either a single-core processor or a multicoreprocessor.

The memory 23 can be used to store any one or more of: the image dataacquired by the IR camera, the image data acquired by the visiblespectrum, program code for execution by the CPU, intermediate dataresulting from computations or calculations by the CPU, or other dataand/or program code. Hence, portions of memory 23 can actually reside inthe CPU 22, the visible spectrum camera 5 or the IR camera 20. Memory 23can be include one or more physical storage devices, which may be orinclude random access memory (RAM), read-only memory (ROM) (which may beerasable and programmable), flash memory, miniature hard disk drive, orother suitable type of storage device, or a combination of such devices.

The communication unit 24 enables the eye tracking device 1 tocommunicate with an external device or system (not shown), such as acomputer or other type of processing device. For example, in certainembodiments, at least some of the eye tracking computations may be doneby an external device (e.g., a personal computer), based on dataacquired by the eye tracking device and transmitted to the externaldevice by the communication unit. This may allow the programming orconfiguration of the CPU 22 to be made much simpler, or it may allow theCPU 22 to be replaced by a much simpler type of controller, or evenomitted entirely from the eye tracking device 1. The communication unit24 can be or include a transceiver that performs wired communication,wireless communication, or both. For example, the communication unit 24can be or include any one or more of: a universal serial bus (USB)adapter, Ethernet adapter, modem, Wi-Fi adapter, cellular transceiver,baseband processor, Bluetooth or Bluetooth Low Energy (BLE) transceiver,or the like, or a combination thereof.

Each IR source 6 of the device can be or include, for example, one ormore light emitting diodes (LEDs) or diffused laser sources. Lasersources can be used in conjunction with TOF principles to provide highquality depth determination, such as for use in gesture recognition. Asdiscussed further below, the illumination by the IR source(s) 6 iscontrolled such that for each shutter window of an imaging frame, theillumination can be set on or off. For embodiments in which there ismore than one IR source 6, the device 1 is able to turn on or off eachsource independently.

FIGS. 3 and 4 illustrate the principle of operation of the eye trackingdevice, according to at least one embodiment. Note that FIG. 3 isintended to be schematic in nature, such that the actual positions ofthe source 6 and sensor 7 in an actual implementation may differ fromtheir positions as shown in FIG. 3. The IR source 6 transmits IR energyat an angle toward one of the lenses 34 of the eyeglasses 2. Asmentioned above, the eyeglasses 2 are assumed to be standard eyeglasses,such that the lens is made of standard optometric-quality glass or anyother material suitable for standard eyeglass lenses. A portion 31A ofthe transmitted IR energy 31 (typically about 5% of it, with glass) isreflected off the inner surface of the nearest lens 34 to reach an eye32 of the user. The remaining portion 31B (typically about 95% of it,with glass) of the transmitted IR energy 31 is transmitted completelythrough the lens to the user's environment. As shown in FIG. 3, at leasta portion 31C of the IR energy that reaches the user's eye is reflectedoff the cornea back to the inner surface of the lens, where a portion31D of that remaining energy is further reflected back to the sensor 7of the IR camera (not fully shown in FIGS. 3 and 4). The IR energy 31Dthat reaches the IR sensor 7 is detected and used by the CPU 22 (oralternatively by an external device) to determine the eye position(i.e., using pupil and/or iris identification). Once the eye position isdetermined, it is possible to identify the gaze location on the RGBvideo image by using standard methods for gaze tracking. One way toaccomplish this is by using a polynomial to map the pupil center or thepupil-glint vector to the RGB coordinates.

Further, at least a portion 31E (FIG. 4) of the IR energy 31B that wastransmitted through the lens to the user's environment is reflected offobjects 41 in the environment back to the IR sensor 7, as shown.Additionally, ambient IR energy 32 in the environment (e.g., fromobjects 41) may also reach the IR sensor 7. However, the fast shutterand TOF principles of the IR camera 20 enable the eye tracking device 1to filter out all of that IR energy except the portion 31D of IR energyreflected from the user's eye that reaches the sensor 7. This can bedone by setting the shutter timing of the IR camera 20 so that IR energytransmitted from the device 1 will be cut off by the shutter (which maybe electronic) on its way back to the IR camera's sensor 7, so that onlyIR reflected or originating from objects from very nearby the sensor 7(e.g., within a few centimeters) is captured by the IR camera 20; thatis, only energy with a sufficiently short TOF is allowed to be captured.This enables the IR camera 20 to capture only the image of the eye,without reflections from outside objects.

Further, by using multiple shutter (time) windows within each imagingframe, it is possible to eliminate reflections from IR light that didnot originate from the device 1, i.e., to filter out ambient IR. One wayto do that is to subtract a non-illuminated image acquired during oneshutter window from an IR-illuminated image acquired during animmediately preceding or immediately following shutter window.

Moreover, by using more than one illuminated shutter window per imagingframe, with different IR sources being active during differentconsecutive shutter windows in each frame, the device 1 can perform orenable robust glint and pupil identification (glint detection can beused for gaze stabilization to allow camera movement on the head withoutcompromising the accuracy of the gaze determination). FIG. 5 illustratesthis principle, for an example IR illumination scheme that uses two IRsources 6, called Source 1 and Source 2. In the illustrated scheme, theIR camera uses three shutter windows per imaging frame, i.e., shutterswindows S0, S1 and S2, where the IR sources are controlled (e.g., by theCPU 22) to be synchronized to the shutter windows, as follows: Duringshutter window S0, Source 1 is on and Source 2 is off; during shutterwindow, S1, source S2 is on and source 1 is off; and during shutterwindow S2, both IR sources are off. Source 1 and source 2 each produce aseparate specular reflection from a different position on the eye, whichcan be used for accurate glint and pupil identification. Hence, the IRcamera 20 in this embodiment actually captures three independent imagesper imaging frame, one for each shutter window, all corresponding toessentially the same instant in time (i.e., the same at least forpurposes of gaze tracking). Those separate images per frame areseparately stored in memory 23 and are independently retrievable toallow processing as described above.

FIG. 6 shows an embodiment of the eye tracking device with twoillumination sources. A second IR source 6B can be positioned on or nearthe eyeglasses frame 3 proximate to the bridge of user's nose, forexample. The second IR source 6B can be connected to the housing of thedevice by a connector 61 such as a wire, which may be rigid orsemi-rigid to enable it to conform to the contour of the lens frame, orby any other suitable connection mechanism.

FIG. 7 is a flow diagram showing an example of the operational processof the eye tracking device, according to some embodiments. At step 701the device 1 transmits IR energy from an IR source 6. At step 702 the IRsensor 7 of the fast shutter IR camera 20 detects infrared energy. TheIR camera 20 then applies one or more gating functions at step 703 tofilter out external reflections of IR from the detected IR, based ontheir TOFs. The IR camera 20 further applies one or more gatingfunctions at step 704 to filter out ambient IR from the detected IR,based on their TOFs. The CPU 23 then determines at step 705 the eyeposition of the user, based on only the nearby reflections of IR energy,e.g., reflections from the user's eye.

The machine-implemented operations described above can be implemented byprogrammable circuitry programmed/configured by software and/orfirmware, or entirely by special-purpose circuitry, or by a combinationof such forms. Such special-purpose circuitry (if any) can be in theform of, for example, one or more application-specific integratedcircuits (ASICs), programmable logic devices (PLDs), field-programmablegate arrays (FPGAs), system-on-a-chip systems (SOCs), etc.

Software or firmware to implement the techniques introduced here may bestored on a machine-readable storage medium and may be executed by oneor more general-purpose or special-purpose programmable microprocessors.A “machine-readable medium”, as the term is used herein, includes anymechanism that can store information in a form accessible by a machine(a machine may be, for example, a computer, network device, cellularphone, personal digital assistant (PDA), manufacturing tool, any devicewith one or more processors, etc.). For example, a machine-accessiblemedium includes recordable/non-recordable media (e.g., read-only memory(ROM); random access memory (RAM); magnetic disk storage media; opticalstorage media; flash memory devices; etc.), etc.

Examples of Certain Embodiments

Certain embodiments of the technology introduced herein are assummarized in the following numbered examples:

1. An eye movement tracking device comprising: a first illuminationsource to transmit energy within a frequency band from a locationproximate to an eye of a person, such that at least a first portion ofthe transmitted energy is reflected off a lens of eyewear worn by theperson to subsequently reflect off the eye of the person and such thatat least a second portion of the transmitted energy is transmittedthrough the lens to reflect off objects in an environment of the person;a time-of-flight detector to detect reflections of at least the firstportion of the transmitted energy, and to distinguish the detectedreflections of the first portion of the transmitted energy from otherenergy detected by the time-of-flight detector in said frequency band,based on times of flight of the reflections of the first portion of thetransmitted energy and said other energy; and a processor configured touse the detected reflections of the first portion of the transmittedenergy to determine a position of the eye of the person.

2. An eye movement tracking device according to example 1, furthercomprising a fastener by which the eye movement tracking device can bedetachably mounted to standard eyeglasses.

3. An eye movement tracking device according to either of examples 1 and2, wherein the transmitted energy is infrared energy.

4. An eye movement tracking device according to any of examples 1through 3, further comprising a human-visible spectrum camera to captureimages of a scene corresponding to a gaze direction of the person.

5. An eye movement tracking device according to any of examples 1through 4, further comprising a second illumination source.

6. An eye movement tracking device according to any of examples 1through 5, wherein the first illumination source is controlled totransmit energy during a first shutter window of an imaging frame andnot to transmit energy during a second shutter window of the imagingframe; and wherein the processor is further configured to: identifyambient energy in the environment based on a difference between energydetected by the time-of-flight detector during the first shutter windowand energy detected by the time-of-flight detector during the secondshutter window; and filter out the ambient energy so that determinationof the gaze direction of the eye is not affected by the ambient energy.

7. An eye movement tracking device according to any of examples 1through 6, wherein the first and second shutter windows do not overlap.

8. An eye movement tracking device according to any of the precedingexamples 1 through 7, further comprising a second illumination sourcecontrolled to transmit energy during a third shutter window of theimaging frame, wherein the first illumination source does not transmitduring the third shutter window and the second illumination source doesnot transmit during the first shutter window.

9. An eye movement tracking device according to any of examples 1through 8, wherein the processor is further configured to: distinguishenergy detected by the sensor during the first shutter window fromenergy detecting by the sensor during the third shutter window; andidentify corneal glints from the eye based on a difference between theenergy detected by the sensor during the first shutter window and theenergy detected by the sensor during the third shutter window.

10. An eye movement tracking device according to any of examples 1through 9, wherein the processor is configured to perform gazestabilization based on the corneal glints.

11. A method of tracking eye movement of a person, the methodcomprising: transmitting energy from a first source located proximate toan eye of the person, so that at least a first portion of thetransmitted energy is reflected off an inner surface of a lens ofeyewear worn by the person and at least a second portion of thetransmitted energy is transmitted through the lens to reflect offobjects in an environment of the person; detecting, by a sensor,reflections of both the first portion and the second portion of thetransmitted energy; using corresponding times of flight of the first andsecond portions of the transmitted energy to distinguish detectedreflections of the first portion of the transmitted energy from detectedreflections of the second portion of the transmitted energy; anddetermining a gaze direction of the eye by using the detectedreflections of the first portion of the transmitted energy and not thedetected reflections of the second portion of the transmitted energy.

12. A method according to example 11, wherein said transmitting and saiddetecting are performed by an apparatus removably mounted to the eyewearworn by the person.

13. A method according to either of the preceding examples 11 and 12,wherein the energy transmitted from the first source compriseselectromagnetic energy in an infrared portion of the electromagneticspectrum and does not include energy in the human-visible portion of theelectromagnetic spectrum.

14. A method according to any of examples 11 through 13, wherein theeyewear comprises conventional prescription eyewear, and the source andthe detector are removably mounted to the eyewear.

15. A method according to any of examples 11 through 14, wherein thetransmitted energy comprises infrared light and does not include energyin a human-visible portion of the electromagnetic spectrum.

16. A method according to any of examples 11 through 15, wherein saidtransmitting energy from the first source comprises transmitting energyfrom the first source during a first shutter window in each of aplurality of sequential imaging frames; wherein the first source doesnot transmit during a second shutter window in each of the plurality ofsequential imaging frames; and the method further comprising:identifying ambient energy in the environment based on a differencebetween energy detected by the sensor during the first shutter windowand energy detected by the sensor during the second shutter window ineach of the plurality of imaging frames; and filtering out the ambientenergy so that said determining the gaze direction of the eye is notaffected by the ambient energy.

17. A method according to any of examples 11 through 16, wherein thefirst and second shutter windows do not overlap.

18. A method according to any of examples 11 through 17, furthercomprising: transmitting energy from a second source during a thirdshutter window in each of the plurality of sequential imaging frames;wherein the first source does not transmit during the third shutterwindow and the second source does not transmit during the first shutterwindow, in each of the plurality of sequential imaging frames; themethod further comprising: distinguishing energy detected by the sensorduring the first shutter window from energy detecting by the sensorduring the third shutter window, in at least one of the plurality ofsequential imaging frames; and identifying corneal glints from the eyebased on a difference between the energy detected by the sensor duringthe first shutter window and the energy detected by the sensor duringthe third shutter window, in said at least one of the plurality ofimaging frames, wherein determining the gaze direction of the eyeincludes using the corneal glints.

19. A method of tracking eye movement, the method comprising:transmitting only infrared light, from a first source mounted oneyeglasses worn by a person, so that at least a first portion of thetransmitted infrared light is reflected off an inner surface of a lensof the eyeglasses and subsequently reflected off an eye of the person,and at least a second portion of the transmitted infrared light istransmitted through the lens to reflect off objects in an environment inwhich the person is located; detecting, by a sensor mounted on theeyeglasses worn by the person, reflections of both the first portiontransmitted infrared light reflected off the eye of the person and thesecond portion of the transmitted infrared light reflected off objectsin the environment; determining corresponding times of flight of thedetected reflections of the first and second portions of the transmittedlight; filtering out detected reflections of the second portion of thetransmitted infrared light based on times of flight of the detectedreflections of the first and second portions of the transmitted infraredlight; filtering out ambient infrared light from light detected by thesensor; and tracking positions of the eye based on filtered outputs ofthe sensor, by using the detected reflections of the first portion ofthe transmitted infrared light.

20. A method according to example 19, wherein said transmitting infraredlight from the first source comprises transmitting infrared light fromthe first source during a first shutter window in each of a plurality ofsequential imaging frames; and wherein the first source does nottransmit during a second shutter window in each of the plurality ofsequential imaging frames, and wherein the first and second shutterwindows do not overlap.

21. A method according to either of examples 19 and 20, furthercomprising:

transmitting infrared light from a second source during a third shutterwindow in each of the plurality of sequential imaging frames, whereinthe first source does not transmit during the third shutter window andthe second source does not transmit during the first shutter window, ineach of the plurality of sequential imaging frames; identifying theambient energy in the environment based on a difference between energydetected by the sensor during the first shutter window and energydetected by the sensor during the second shutter window, in each of theplurality of imaging frames; distinguishing energy detected by thesensor during the first shutter window from energy detecting by thesensor during the third shutter window, in at least one of the pluralityof sequential imaging frames; and identifying corneal glints from theeye based on a difference between the energy detected by the sensorduring the first shutter window and the energy detected by the sensorduring the third shutter window, in said at least one of the pluralityof imaging frames, wherein said tracking positions of the eye includesperforming gaze stabilization based on the corneal glints.

22. A device for tracking eye movement of a person, the devicecomprising: means for transmitting energy from a first source locatedproximate to an eye of the person, so that at least a first portion ofthe transmitted energy is reflected off an inner surface of a lens ofeyewear worn by the person and at least a second portion of thetransmitted energy is transmitted through the lens to reflect offobjects in an environment of the person; detecting, by a sensor,reflections of both the first portion and the second portion of thetransmitted energy; means for using corresponding times of flight of thefirst and second portions of the transmitted energy to distinguishdetected reflections of the first portion of the transmitted energy fromdetected reflections of the second portion of the transmitted energy;and means for determining a gaze direction of the eye by using thedetected reflections of the first portion of the transmitted energy andnot the detected reflections of the second portion of the transmittedenergy.

23. A device according to example 22, wherein the device is removablymountable to the eyewear worn by the person.

24. A device according to either of the preceding examples 22 and 23,wherein the energy transmitted from the first source compriseselectromagnetic energy in an infrared portion of the electromagneticspectrum and does not include energy in the human-visible portion of theelectromagnetic spectrum.

25. A device according to any of examples 22 through 24, wherein theeyewear comprises conventional prescription eyewear, and the source andthe detector are removably mounted to the eyewear.

26. A device according to any of examples 22 through 25, wherein thetransmitted energy comprises infrared light and does not include energyin a human-visible portion of the electromagnetic spectrum.

27. A device according to any of examples 22 through 26, wherein saidmeans for transmitting energy from the first source comprises means fortransmitting energy from the first source during a first shutter windowin each of a plurality of sequential imaging frames; wherein the firstsource does not transmit during a second shutter window in each of theplurality of sequential imaging frames; and the device furthercomprising: means for identifying ambient energy in the environmentbased on a difference between energy detected by the sensor during thefirst shutter window and energy detected by the sensor during the secondshutter window in each of the plurality of imaging frames; and means forfiltering out the ambient energy so that said determining the gazedirection of the eye is not affected by the ambient energy.

28. A device according to any of examples 22 through 27, wherein thefirst and second shutter windows do not overlap.

29. A device according to any of examples 22 through 28, furthercomprising: means for transmitting energy from a second source during athird shutter window in each of the plurality of sequential imagingframes; wherein the first source does not transmit during the thirdshutter window and the second source does not transmit during the firstshutter window, in each of the plurality of sequential imaging frames;the device further comprising: means for distinguishing energy detectedby the sensor during the first shutter window from energy detecting bythe sensor during the third shutter window, in at least one of theplurality of sequential imaging frames; and means for identifyingcorneal glints from the eye based on a difference between the energydetected by the sensor during the first shutter window and the energydetected by the sensor during the third shutter window, in said at leastone of the plurality of imaging frames, wherein determining the gazedirection of the eye includes using the corneal glints.

30. A device of tracking eye movement, the device comprising: means fortransmitting only infrared light, from a first source mounted oneyeglasses worn by a person, so that at least a first portion of thetransmitted infrared light is reflected off an inner surface of a lensof the eyeglasses and subsequently reflected off an eye of the person,and at least a second portion of the transmitted infrared light istransmitted through the lens to reflect off objects in an environment inwhich the person is located; means for detecting, by a sensor mounted onthe eyeglasses worn by the person, reflections of both the first portiontransmitted infrared light reflected off the eye of the person and thesecond portion of the transmitted infrared light reflected off objectsin the environment; means for determining corresponding times of flightof the detected reflections of the first and second portions of thetransmitted light; means for filtering out detected reflections of thesecond portion of the transmitted infrared light based on times offlight of the detected reflections of the first and second portions ofthe transmitted infrared light; means for filtering out ambient infraredlight from light detected by the sensor; and tracking positions of theeye based on filtered outputs of the sensor, by using the detectedreflections of the first portion of the transmitted infrared light.

31. A device according to example 30, wherein said means fortransmitting infrared light from the first source comprises means fortransmitting infrared light from the first source during a first shutterwindow in each of a plurality of sequential imaging frames; and whereinthe first source does not transmit during a second shutter window ineach of the plurality of sequential imaging frames, and wherein thefirst and second shutter windows do not overlap.

32. A device according to either of examples 30 and 31, furthercomprising:

means for transmitting infrared light from a second source during athird shutter window in each of the plurality of sequential imagingframes, wherein the first source does not transmit during the thirdshutter window and the second source does not transmit during the firstshutter window, in each of the plurality of sequential imaging frames;means for identifying the ambient energy in the environment based on adifference between energy detected by the sensor during the firstshutter window and energy detected by the sensor during the secondshutter window, in each of the plurality of imaging frames; means fordistinguishing energy detected by the sensor during the first shutterwindow from energy detecting by the sensor during the third shutterwindow, in at least one of the plurality of sequential imaging frames;and means for identifying corneal glints from the eye based on adifference between the energy detected by the sensor during the firstshutter window and the energy detected by the sensor during the thirdshutter window, in said at least one of the plurality of imaging frames,wherein said tracking positions of the eye includes using the cornealglints to perform gaze stabilization.

Any or all of the features and functions described above can be combinedwith each other, except to the extent it may be otherwise stated aboveor to the extent that any such embodiments may be incompatible by virtueof their function or structure, as will be apparent to persons ofordinary skill in the art. Unless contrary to physical possibility, itis envisioned that (i) the methods/steps described herein may beperformed in any sequence and/or in any combination, and that (ii) thecomponents of respective embodiments may be combined in any manner.

Although the subject matter has been described in language specific tostructural features and/or acts, it is to be understood that the subjectmatter defined in the appended claims is not necessarily limited to thespecific features or acts described above. Rather, the specific featuresand acts described above are disclosed as examples of implementing theclaims and other equivalent features and acts are intended to be withinthe scope of the claims.

What is claimed is:
 1. A method of tracking eye movement, the methodcomprising: transmitting only infrared light, from a first sourcemounted on eyeglasses worn by a person, so that at least a first portionof the transmitted infrared light is reflected off an inner surface of alens of the eyeglasses and subsequently reflected off an eye of theperson, and at least a second portion of the transmitted infrared lightis transmitted through the lens to reflect off objects in an environmentin which the person is located; detecting, by a sensor mounted on theeyeglasses worn by the person, reflections of both the first portiontransmitted infrared light reflected off the eye of the person and thesecond portion of the transmitted infrared light reflected off objectsin the environment; determining corresponding times of flight of thedetected reflections of the first and second portions of the transmittedlight; filtering out detected reflections of the second portion of thetransmitted infrared light based on times of flight of the detectedreflections of the first and second portions of the transmitted infraredlight; filtering out ambient infrared light from light detected by thesensor; and tracking positions of the eye based on filtered outputs ofthe sensor, by using the detected reflections of the first portion ofthe transmitted infrared light.
 2. A method according to claim 1,wherein said transmitting infrared light from the first source comprisestransmitting infrared light from the first source during a first shutterwindow in each of a plurality of sequential imaging frames; and whereinthe first source does not transmit during a second shutter window ineach of the plurality of sequential imaging frames, and wherein thefirst and second shutter windows do not overlap; the method furthercomprising: transmitting infrared light from a second source during athird shutter window in each of the plurality of sequential imagingframes, wherein the first source does not transmit during the thirdshutter window and the second source does not transmit during the firstshutter window, in each of the plurality of sequential imaging frames;identifying the ambient energy in the environment based on a differencebetween energy detected by the sensor during the first shutter windowand energy detected by the sensor during the second shutter window, ineach of the plurality of imaging frames; distinguishing energy detectedby the sensor during the first shutter window from energy detecting bythe sensor during the third shutter window, in at least one of theplurality of sequential imaging frames; and identifying corneal glintsfrom the eye based on a difference between the energy detected by thesensor during the first shutter window and the energy detected by thesensor during the third shutter window, in said at least one of theplurality of imaging frames, wherein said tracking positions of the eyeincludes using the corneal glints to perform gaze stabilization.
 3. Amethod according to claim 1, further comprising: using a human-visiblespectrum camera to capture images of a scene corresponding to a gazedirection of the person.
 4. A method according to claim 1, furthercomprising: transmitting light from a second source mounted oneyeglasses.
 5. A method according to claim 1, wherein said transmittingcomprises controlling the first source to transmit energy during a firstshutter window of an imaging frame and not to transmit energy during asecond shutter window of the imaging frame; and wherein said filteringout ambient infrared light from light detected by the sensor comprisesidentifying ambient infrared light in the environment based on adifference between infrared light detected by a time-of-flight detectorduring the first shutter window and infrared light detected by thetime-of-flight detector during the second shutter window.
 6. A methodaccording to claim 5, wherein the first and second shutter windows donot overlap.
 7. A method according to claim 5, further comprising:controlling a second source to transmit energy during a third shutterwindow of the imaging frame; wherein the first source does not transmitduring the third shutter window and the second source does not transmitduring the first shutter window.
 8. A method according to claim 7,further comprising: distinguishing energy detected by the sensor duringthe first shutter window from energy detecting by the sensor during thethird shutter window; and identifying corneal glints from the eye basedon a difference between the energy detected by the sensor during thefirst shutter window and the energy detected by the sensor during thethird shutter window.
 9. A method according to claim 1, furthercomprising: detecting corneal glints; and performing gaze stabilizationbased on the detected corneal glints.